d.) Remarks 

Turning first to Item 5 of the instant Action, applicant will attempt to 
comply with this request by addressing each numbered item in the Action in serial 
order. 

Item 1 is self-evident. Item 2, the response to arguments and discussion in 
applicant's previous amendment, are noted. Applicant will attempt herein to more 
clearly set forth the distinctions of the invention over the references by using more 
specific language in the claims. Item 4 requires no response, and Item 5 is noted 
above. 

Items 6 and 7 state a rejection under § 1 12 that cites several different 
phrases used in the claims and requests applicant to locate antecedents in the 
specification and drawings. In serial order, they are (application citations in 
italics): 

a) " determine the existence and number and ang les of vertices in a line 
which could be drawn between said points.' 9 Note page 29, lines 3-6: "The hand 
drawn entry is analyzed geometrically. The software goes through and looks for 
recognizable vertices that exceed an angular threshold. The number of vertices 
that are detected is stored for later use. " Note also page 38, lines 14-24: "As 
shown in Figures 10 and 11, a vertex determination is made if the slice angle 
(current deltangle) exceeds the current threshold angle, AND the current 
deltangle value is less than the previous deltangle value. This latter factor 
indicates that a region of maximum curvature has been detected. Note that the 
slice of Figure 10 has an angle of 58°, and the subsequent slice (Figure 11 ) has a 
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slice angle of5° f indicating that the software steps have iterated along the stroke 
past the region of maximum angle, which is at (or close to) the midpoint of the 
slice of Figure JO. Therefore, the software places the vertex at the midpoint of the 
slice of Figure 10. The process continues until all slices of the stroke have been 
processed, and all vertices determined. " These citations provide antecedent 
support that is more than adequate. 

b) "color rules to determine agglomeration of said entries as a single 
entity Note page 28, line 18 to page 29, line "It can be determined that only 
intersecting hand drawn entries or hand drawn entries in close proximity that are 
of the exact same color can be added together or joined to comprise a composite 
recognizable object. Furthermore, color can be a determining factor for 
agglomeration. For instance, if a stroke is added to an existing object where the 
stroke and object are the same color, agglomeration can take place. However, if 
the added stroke is not the same color as the existing object, agglomeration will 
not take place if it has been set up that only the agglomeration of same color hand 
drawn entries can occur. " This is an explicit explanation of the color rule 
regarding agglomeration. Note the questioned phrase was in the claim as 
originally filed. 

c) "excluding identification of shapes that do not conform to said set of 
rules regarding maximum proximate distance to said another graphic 
object" Note page 5, lines 5-18: "Agglomeration is the ability to draw an object 
and THEN LATER at any time in the future, draw another stroke (or strokes) that 
can be recognized as an "additional stroke(s) 9y added to the original stroke. The 
Jaeger Patent Application 

METHOD FOR CREATING AND OPERATING CONTROL SYSTEMS 

Second Amendment R2 
Pf»ap?.S of 40 



added strokes are analyzed together with the original strokes to redefine the 
identity of the original object. The agglomeration process is subject to a proximity 
default gap consideration. That is, the distance from the u additional stroke" (or 
strokes) to the original object (stroke) determines whether this software considers 
the second stroke(s) as an additional stroke to be added to the original stroke for 
the purpose of modifying its recognition as an object. . . One practical default gap 
for this purpose could be 12 pixels (approximately 1/8 inch on a standard VGA 
monitor). " 

Note also page 31-32, lines 23-4: u Default gap: A user-defined distance 
(measured in any unit a user desires, i.e., pixels, millimeters, inches, picas, etc.) 
that comprises the maximum distance that can exist as a gap in a hand drawn 
object's defining stroke or strokes. This default gap is also the maximum 
permissible distance that any hand drawn entry can be from any previously 
existing hand drawn entry in order for a newly hand drawn entry to be considered 
part of a previously existing hand drawn entry. " The proximity default gap 
consideration was expressed in original claim 27 as "maximum proximate 
distance" and is thus fully supported by the original disclosure. 

d) "reiterating said slice step*" Note page 33, line 13 to page 34, line 6: 
"Slice: The slice is the selection of points which is used to measure the angle of a 
drawn stroke and determine if there is any deviation. A slice is a "theoretical" 
line constructed point-to-point between three adjacent points (each defined by the 
pixel threshold), which are required to analyze a deviation in a stroke. The 
distance between the first point and the second point must exceed the "pixel 
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threshold", defined above. Likewise for the distance between the second point and 
the third point. These three points comprise a slice. The length is analyzed from 
point to point accounting for individual displacements of each point along the 
way, which may not create a perfectly straight "theoretical" line. With regard to 
Figure 4, the three points shown are not in a straight line, which is typically the 
case with a real hand drawn entry. So the total length of the slice is comprised of 
the distance between point one and point two added to the distance between point 
two and point three. Note that a slice is a transitory collection of points which are 
used to measure an angle of deviation. In contrast, "segments" continue to exist 
in the software as part of a hand drawn object. 

The routine for identifying each slice is depicted in Figure 5. Note that the 
three points selected to comprise each slice are spaced apart by a distance greater 
than the pixel threshold. " 

Note also page 38, line 22-23: "The process continues until all slices of the 
stroke have been processed, and all vertices determined. " Thus the recitation of a 
slice step and its reiteration is clearly supported. 

Item 8 questions the relative meanings of "substantially orthogonal", etc. in 
the claims. Note page 73, lines 18-20: "The algorithm looks for the difference 
between the end-to-end segment angles first - it measures the angle between 
segments and looks for angles that are greater than 65 and less than 110. This is 
the initial test. " The person having ordinary skill knows that a rectangle is 
comprised ideally of four right angles, and the range of 65°- 1 10° is a fair 
definition of 'substantially orthogonal 5 as well as 'substantially non-orthogonal \ 
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Items 9-1 1 state an objection to the phrase "maximum total reduction" in 
claim 96. This claim has been canceled. 

Item 12 states an objection to claims 38 and 47. It is believed that claim 38 
is fully supported, as indicated in the reply to item 7(d) above. Claim 47 was 
previously canceled and not presented for examination. 

Items 13 and 14 state a rejection under §102 of claims 1, 2, 4-7, 9-15, 24, 
29-43, and 48-49 over Yamakawa, previously cited. Items 15-17 state a rejection 
of most other claims based on Yamakawa, Capps, and Meeks. Since many claims 
have been amended herein, it would not be appropriate to argue the old rejections 
per se. Rather, these references and the distinctions of the claims thereover will be 
discussed in the order of claim presentation. 

Claim 1 has been amended to add a more explicit recitation of the Wide 
Pen test, including constructing a minimal rectangle about a hand drawn entry, 
placing an invisible trial object of defined pixel width lines in the minimal 
rectangle, counting the points of the hand drawn entry that coincide with the wide 
line, and declaring an identification if the number of points is greater than a 
predetermined threshold. This recitation is supported in the specification, inter 
alia, on pages 40-4 L Claims 15 and 17, which were similar in scope, have been 
canceled. 

The Wide Pen Test recitations of the previous claims stand rejected under 
§103 over the combination of Yamakawa and Capps. Capps does generate a 
bounding rectangle, as shown in Figure 14 of the reference. However, this 
bounding rectangle is not used in any way for stroke analysis. Rather, the 
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bounding box is used to locate the sequential frames of a brief animation that 
graphically demonstrates the erasure of an object from the display. There is no 
basis for assuming that the Capps bounding box bears any relationship to stroke 
analysis. 

The current rejection cites Capps col. 10, lines 6-21 as a showing of 
adaptively altering the width of the pen stroke in the Wide Pen test. Capps does 
not have any such showing of a Wide Pen test, nor of adaptively altering a pen 
stroke for analysis and identification purposes. The reference undertakes to reduce 
the number of line segments by removing segments that are shorter than a 
predetermined LENGTH threshold, and then eliminating corners that are less than 
a predetermined ANGLE to reduce the figure substantially and then count the 
remaining comers. There is no disclosure regarding pen WIDTH, nor of 
determining the coincidence of points of a drawn input to a machine-rendered 
regular figure drawn in a wide pen line. 

Capps col. 11, lines 15-26 is cited as a showing of analysis by determining 
coincidence of the drawn input with an identifiable geometric shape (the Wide Pen 
test). This rejection is completely unsupported by the facts. The Capps citation 
relates to an analysis technique in which the drawn input is processed to find 
comer points by first drawing an imaginary straight line between the first and last 
points of the input, and then finding the farthest point on the drawn input from the 
imaginary line. If the farthest point is more than a predetermined distance 
(MaxDev) from the imaginary line, it is defined as a comer point The process is 
reiterated until a set of processed points is developed and none of the actual points 
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on the input stroke are farther that the predetermined distance from the imaginary 
line. 

It is significant to note that Capps never describes inscribing a trial object 
in the bounding rectangle, nor determining the coincidence of the points of the 
hand drawn entry with the lines of the trial object to determine if a minimum 
coincidence level has been reached. The imaginary lines drawn by Capps in his 
bounding rectangle are used only to calculate the most distant point therefrom to 
designate as a comer, NOT to determine coincidence with a known trial object. 
There is not a sufficient factual foundation in Capps for rejecting the Wide Pen 
Test feature, and claim 1 should be allowed. 

Applicant feels compelled to state for the record that other recited features 
of claim 1 are not met by the prior art citations, as asserted in the rejection. For 
example, Yamakawa col. 7, lines 27-51 is cited as a teaching of the step of 
determining the existence and number and angles of vertices in a line drawn 
between the points of the hand drawn entry. This passage describes how 
Yamakawa divides any input character into 32 equal segments (regardless of size 
or shape) and places a dot at the end of each segment. It then connects the dots, 
and determines the angles of the two lines extending from each dot. These dots 
are arbitrarily located according to the divide-by-32 process, and are not vertices. 
Indeed, the chance that one of the dots actually is located at a vertex is extremely 
remote. 

In further techniques, Yamakawa describes that each input stroke is broken 
down into component vectors, and that virtual transition strokes may also be 
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reduced to component vectors, after which the vector information is reduced to 
fuzzy vectors. Here it is important to note that the number of component vectors 
is limited in the Yamakawa system. As noted in col. 5, lines 25-30, " In the 
system employing the fuzzy vector, since the total number of the "component 
vectors" is the smallest number of units representing the trajectory of handwriting 
in forming a handwritten character, a number greater than the number of strokes of 
the handwritten character is required for extracting information of the trajectory of 
handwriting therefrom." Note also col. 5, lines 39-43: "That is, the number of the 
component vectors is not changed by the length of the total strokes, but the 
number of the component vectors per one stroke is univocally (sic) determined by 
the number of strokes." Thus the longer the stroke (or combination of strokes), the 
greater the spacing of points in Yamakawa and the less accurate is the analysis. 

It must be emphasized that the use of transition strokes, the use of vectors, 
the limitations on the number of vectors (comparable to, in the present invention, 
the number of data points or slice calculations), or the limitation of vector numbers 
based on the number of strokes in a character, are all concepts that are completely 
antithetical to the techniques described and claimed in the present invention. 
There is no actual determination of vertices or angular trends in Yamakawa; 
instead, the reference resorts to 'fuzzy vectors', which are best understood as 
broad categorizations of the angular relationships between the adjacent limited 
number of component vectors of Yamakawa. The fuzzy vectors are integers that 
describe the component vector changes only according to which Cartesian 
quadrant the change is pointing toward, and thus provides no more angular 
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resolution than ±90°. This technique cannot determine the presence or location of 
a vertex. Rather, the fuzzy vector information is compared to a dictionary of fuzzy 
vector patterns that correlate with known characters. 

Applicant has abundantly supported the claim language regarding 
determining the existence, number and angles of the vertices of a drawn entry, 
particularly how a vertex is measured and found among all the points of a hand 
drawn entry. 

Claims 2, 4-8 depend from claim 1 and further refine that patentable 
recitation, and should also be allowable. 

Claim 9 recites the invention in a mode similar to claim 1, but instead of the 
Wide Pen Test it specifies a plurality of hand drawn entries of various colors, and 
that the set of rules for identification includes rules for agglomerating entries of 
the same color. Strangely, no reference or combination of references has been 
applied to claim 9 in the Second Action. Indeed, none of the citations deals with 
color in the handling of hand drawn entries, and it is clear that claim 9 is allowable 
over the art. 

Claim 10 recites the invention in a mode similar to claims 1 and 9, and also 
adds the step of using arrow logics to establish attributes of the identified shape. 
This claim stands rejected over Yamakawa only, and this reference makes no 
showing of arrows in any use or function that is similar to arrow logics. Thus 
claim 10 is prima facie allowable over the reference. Perhaps the rejection should 
have made reference to Capps Figures 3A-3E, which are cited in rejecting claims 
23 and 95. Capps Figures 3 A-3E are cited to show the step of drawing at least one 
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arrow from an attribute shown in an info window to at least one identified shape 
outside said info window. These figures make no such showing. Rather, Figures 
3A-3E depict a scrub (erasure) gesture drawn over a sentence of text, or one side 
of a graphic object, or the vertex of a graphic object, or a caret gesture made 
between two characters of text, or an open space gesture made between two lines 
of text (col. 3, lines 28-38). Although the graphic object depicted is an arrow, this 
arrow has absolutely no function (programming or otherwise) other than passively 
receiving the scrub gesture. It does not convey any attribute from an info window 
to an identified shape outside the info window, as set forth in the definition of 
arrow logics on page 13 of the specification. Therefore claim 10 should be 
allowed. 

Claim 1 1 also employs the same general format as the previous independent 
claims, and more specifically recites the step of creating an info window for at 
least one of said identified shapes, said info window enabling setting and altering 
attributes for said at least one identified shape. This claim stands rejected over 
Yamakawa under §102, although no specific citation in the reference is applied to 
the claim. There being no info window disclosure in this (or any other) reference, 
it is asserted that claim 1 1 should be allowed. 

Claims 12-14, 16, and 18-22 all depend from claim 1, further refining that 
patentable recitation, and thus should also be allowable. Note also that claim 18 
explicates one way for reiterating the Wide Pen Test when a coincidence 
comparison is unsuccessful, by substituting a further trial object in the bounding 
rectangle. Claim 19 explicates another way for reiterating the Wide Pen Test by 
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increasing the width of the wide pen stroke when a coincidence comparison is 
unsuccessful. None of these techniques are disclosed in the prior art, and these 
claims should also be allowed. 

Claim 23 depends from claim 1 1, and further specifies the step of drawing 
at least one arrow from an attribute shown in an info window to at least one 
identified shape outside said info window. Capps Figs. 3A-3E is cited in 
particular in rejecting this claim. These figures make no showing regarding the 
claimed invention. Claim 1 1 recites inter alia an info window, and claim 23 adds 
the use of an arrow to convey an attribute shown in an info window to an 
identified shape outside the info window. Figs. 3A-3E of the reference portray the 
steps in drawing and recognizing a scrub gesture and applying its function to an 
underlying onscreen object. There is no teaching of the use of an info window for 
any purpose, nor is there any teaching of drawing an arrow from an info window 
to an identified onscreen object to convey an attribute from the info window to the 
object. Thus it is clear that the claim defines the invention over the reference and 
should be allowed. 

Claims 24 and 25 depend from allowable claim 1, and further add the step 
of determining the angular orientation of the hand drawn entry with respect to a 
reference orientation, and (claim 25) thereafter excluding identification of shapes 
that do not conform to the rules regarding angular orientation. The Yarnakawa 
reference is cited against claim 24, particularly Figs. 1 1-14 and col. 7-8 of the 
reference. These figures depict the coarse angular resolution of Yarnakawa due to 
the use of fuzzy vectors which reduce lines to membership values in Cartesian 
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quadrants. Note col. 8, line 38 et seq.: "Through the above described procedures, 
four membership grades pig, f4 2 , and }4 3 are output from one component vector 
and all of the patterns of the membership grades ji 0 , pi 2 , and ]4 3 are stored in 
memory. 
Step 7 

Taking up, as the objects, the dictionary's characters with the same number of 
strokes as the handwritten input character, the membership grades of the 
handwritten input character obtained in step 6 are matched with membership 
grades of all of the dictionary's characters to thereby calculate similarity 
measure..." Clearly Yamakawa derives the membership grades as a numerical 
derivative of the vector, and then stores the membership grades of various 
characters and matches them to make an identification. This numerical matching 
technique has no relevance to the task of actually determining the angular 
orientation of the hand drawn entry, and claim 24 should be allowed. 

It is noted for the record that claim 25 stands rejected under §1 12, but there 
is no rejection of the claim over the prior art. Given the literal meaning of § 102 
("A person shall be entitled to a patent unless...") and the fact that the rejection 
under §1 12 has been overcome by this amendment, it is asserted that claim 25 is 
factually, procedurally, and lawfully allowable and should be issued. 

Claim 26 depends from allowable claim 1, and further refines that 
patentable recitation by adding the step of assessing the proximity of the entry to 
another graphic object to exclude identification of shapes that do not meet rules 
regarding maximum proximate distance. Capps Figure 4 is cited against this 
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feature. As noted previously, Capps does determine the proximity of a hand 
drawn entry to a previous onscreen graphic, but it is not for the purpose of 
identification of the onscreen object (by way of exclusion of shapes). Rather, 
Capps determines sufficient proximity in order to apply the gesture (scrub or 
erase) to the proximate object. This proximity does not aid in recognizing the 
scrub gesture, nor in identifying the previous onscreen graphic. Thus the reference 
is not carrying out the claimed process, and claim 26 should be allowed as well. 

Claim 29 follows the format of other independent claims and notably 
includes the step of identifying a portion of the hand drawn entry drawn more 
slowly than other portions of the hand drawn entry. It stands rejected only over 
Yamakawa, which is not applied explicitly to the claim. The reference makes no 
disclosure regarding changing the speed at which a portion of an entry is 
identified, and claim 29 is clearly allowable over the reference. 

Claims 30-37 depend from claim 29 and further refine that patentable 
recitation and are thus also allowable. In addition, claim 30 recites the step of 
determining the existence of a vertex in the portion of said hand drawn entry, and 
calculating the vertex angle. The Yamakawa citation does not teach or suggest this 
step; rather, it assigns spaced dots along the drawn line and uses them as markers 
for segments. Claim 32 adds that if a vertex angle in the portion is orthogonal, 
there is an increased potential for identifying a rectilinear shape. These analysis 
steps are not taught nor suggested in the prior art, and these claims should be 
allowed. 
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Claim 33 depends from claim 30 and states that if a vertex angle in the 
portion of the hand drawn entry is non-orthogonal, there is increased potential for 
exclusion of all rectilinear shapes. Claim 34 depends from claim 30 and adds that 
if a pair of vertex angles in the portion of the hand drawn entry are substantially 
orthogonal, proximate, and opposite, there is increased potential for identification 
of a folder shape. Yamakawa Figure 9 cannot in any way be construed to teach 
these concepts. These drawn entry analysis steps are not taught nor suggested in 
the prior art, and these claims should be allowed. 

Claim 35 also depends from claim 30 and recites one of the golden clue 
techniques: a vertex identified and measured in a first-drawn portion of a hand 
entry will be more accurately drawn that the latter portions. Thus making such an 
identification may yield an increased potential for identification of a particular 
shape, or exclusion of particular shapes. Yamakawa Fig. 1(b) or Fig. 4 are 
completely irrelevant to this identification technique, and claim 35 should be 
allowed. Claims 36 and 37 depend from claim 35 and further add the limitations 
discussed with reference to claims 32 and 33, and should likewise be allowable. 

Yamakawa uses a vector process that does not process substantially all the 
points of the drawn entry. Rather, it a priori limits drastically the number of 
points to process in accordance with the number of strokes to approximately 32, 
which is typically thousands of times less than the number of points processed in 
the claimed invention. Moreover, Yamakawa does not process accumulated angle 
information to determine curvature of the line, nor does it determine the presence 
of vertices, particularly when the drawn line does not form a clear vertex (abrupt 
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turning point). The rejection points to col. 7, lines 25-67 of Yamakawa, which 
details how the absolute angles of the component vectors are subsequently reduced 
to fuzzy vectors. As noted above, the fuzzy vectors are low order integers that 
represent the grossest approximation of the directions of the component vectors 
from which they are derived. These fuzzy vectors are then compared to a 
dictionary of symbols; they are NOT further processed to identify vertices, 
determine angular trend, or any other process that is employed in the present 
invention. Only the slice processing techniques of the present invention can 
accomplish these tasks, and then use the results to identify the drawn figure with a 
very high degree of accuracy. 

Claims 38-41 set forth this process, and, finding no counterpart in the prior 
art, define patentable subject matter. Note claim 38 has been substantially 
amended to set forth the details of identifying the three points of each slice and 
measuring the angle constructed therebetween. Note that the minimum pixel 
distance between slice points may process a great number of points, and will 
process more points in a longer hand drawn entry. In contrast, Yamakawa sets a 
fixed number of segments to break the drawing into, no matter how long it may 
be, and the longer the drawing the worse will be the resolution of a fixed number 
of segments. 

In the instant invention, the stroke of a hand drawn input is broken down into 
slices of sequences of three serial points that are spaced apart greater than a 
minimum distance, and each slice is analyzed to determine its angle. There is no 
limitation to the number of slices that are processed . The sequential slice angles 
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are analyzed to determine the presence or absence of an angular trend in the 
stroke, and to determine the presence or absence of a vertex. 

Claims 48-50 depend from allowable claim 1 and are thus likewise 
patentable. Regarding the rejection of claim 50, applicant finds no citation in 
Capps "where numeric information can be entered into the pen-based computer 
system," Nor is there any discussion of magic number values in either reference. 

Claim 95 depends from claim 1 1 and adds an arrow logic limitation that is 
the converse of claim 23; that is, the arrow is drawn tQ an attribute in an info 
window from an identified shape outside the info window. The comments 
regarding claim 23 are equally applicable here, and it is asserted that claim 95 
clearly defines the invention over the Capps disclosure regarding arrows, as well 
as any combination that could be made with Yamakawa. 

The claims have been substantially amended to particularly point out the 
manifest distinctions of the invention over the cited art, and many claims have 
been canceled. It is asserted that all claims now presented are allowable, and the 
application is in condition to be passed to issue. Action toward that end is 
earnestly solicited. 
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Respectfully Submitted, 




/Harris Zimmpmian, Esq. 
Registi^rNo. 16, 437 
Attorney for Applicant 
Law Offices of Harris Zimmerman 
1330 Broadway, Suite 710 
Oakland, California 94612 
(510) 465-0828 
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